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1  Introduction 

Our  project  rests  on  the  belief  that  computer  animation  in  the  form  of  narrated  animated 
simulations  can  provide  an  engaging,  effective  euid  flexible  medium  for  instructing  agents  of 
varying  capabilities  to  perform  tasks  that  make  varying  demands  in  workplaces  of  varying  layout. 
To  this  end,  we  have  been  designing  and  implementing  an  integrated  system  which  combines 

•  animated  agents  which  can  demonstrate  the  behavior  to  be  emulated; 

•  automatic  generation  of  appropriate  Natural  Language  narration  which  can  explain  what 
is  being  done  and  why. 

To  date,  our  primary  concern  with  Natural  Language  has  been  as  input  to  the  system,  in 
line  with  the  strong  claim  we  make  in  [1]  that  moving  task  animation  beyond  direct  graphical 
manipulation  forces  one  to  Natural  Language  as  the  only  instruction  source  accessible  to  other 
users  than  the  current  community  of  manually  skilled  (or  programming-wise)  animators.  (To 
this  end,  we  have  been  analysing  constructions  commonly  found  in  NL  instructions,  in  terms  of 
their  representational  requirements  [3]. 

However  here  our  point  of  discussion  is  NL  Generation.  What  makes  us  such  eager  con¬ 
sumers  of  advances  and  technology  in  this  area  is  that  animated  simulations  without  narration 
(ultimately,  spoken  narration)  is  only  half  the  story.  As  researchers  studying  plan  inference 
have  shown  [2],  it  may  be  well-nigh  impossible  to  infer  an  agent’s  intentions  simply  by  observing 
his  or  her  actions  alone. ^  And  we  know  that  the  ability  to  perform  an  action  effectively  in  a 
range  of  environments  requires  understanding  its  intention,  not  just  the  physical  motions  used 
in  some  performance.  Thus,  communicating  intentions  is  as  important  to  effective  teisk  instruc¬ 
tion  as  demonstrating  physical  skills.  Sharing  the  burden  of  communication  between  Natural 
Language  and  graphics,  cis  Feiner  and  McKeown  have  noted  [4],  takes  advantage  of  the  best  of 
both  possible  worlds. 

While  some  parts  of  our  system  are  further  along  than  others,  no  work  at  all  has  yet  been 
done  on  generation.  However,  we  have  tried  to  take  account  of  the  needs  of  generation  in 
designing  the  system,  so  that  we  will  not  have  painted  ourselves  in  a  hole  from  the  start.  We 
clearly  and  hope  to  get  further  ideeis  and  direction  from  this  meeting.  Basically,  the  system  has 
been  designed  so  that  the  generator  will  receive  information  from  three  sources  (see  Figure  1.): 

^This  research  is  partially  supported  by  Lockheed  Engineering  ^lnd  Management  Services  (NASA  Johnson 
Space  Center),  NASA  Ames  Grant  NAG-2-426,  FMC  Corporation,  Martin-Meirietta  Denver  Aerospace,  NSF 
CISE  Grant  CDA88-22719,  and  ARO  Grant  DAAL03-89-C-0031  including  participation  by  the  U.S.  Army  Human 
Engineering  Laboratory. 

^Exaggerating  behavior  to  make  it  more  communicative  may  have  the  adverse  effect  of  meiking  it  less  veridiceil, 
a  situation  inversely  turned  advantageous  by  skilled  cartoon  animators  [5]. 
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•  the  partial  global  plan  (an  incrementally  computed  description  of  what  the  animated  agent 
is  meant  to  do  and  why); 

•  the  basic  animation  commands  (for  particulars  of  what’s  happening  “now”) 

•  the  visualization  plan  (for  what  can  the  viewer  see). 

The  resulting  narrative  is  thus  meant  to  satisfy  the  joint  communicative  goals  of  providing 
an  overall  context  in  which  to  view  the  events  on  the  “screen”  and  explaining  the  reasons  for 
particular  events  that  are  happening,  thereby  transcending  the  merely  visible  portion  of  any 
event,  to  augment  and  reinforce  observable  behavior.  For  a  more  detailed  description  of  the 
system  and  further  discussion  of  instructions  and  task  performance,  the  reader  is  referred  to  [1]. 


References 

[1]  Norman  Badler,  Bonnie  Webber,  Jeff  Esakov  and  Jugal  Kalita.  Animation  from  Instruc¬ 
tions.  Making  Them  Move:  Mechanics,  Control  and  Animation  of  Articulated  Figures. 
Morgan-Kaufmann,  1990.  (Also  appears  as  Technical  Report  CIS-90-17,  Dept,  of  Com¬ 
puter  and  Information  Science,  Univ.  of  Pennsylvania,  Philadelphia  PA,  1990.) 

[2]  Phil  Cohen.  The  Need  for  Referent  Identification  as  a  Planned  Action.  Proc.  of  Interna¬ 
tional  Joint  Conference  on  Artificial  Intelligence,  August  1981,  pp.  31-36, 

[3]  Bonnie  Webber  and  Barbara  Di  Eugenio.  Free  Adjuncts  in  Natural  Language  Instructions. 
Proc.  of  COLING-90.  University  of  Helsinki,  Finland.  August  1990. 

[4]  Feiner,  S.  and  McKeown,  K.  Coordinating  Text  and  Graphics  in  Explanation  Genera¬ 
tion.  Proc.  ARPA  Speech  and  Natural  Language  Workshop,  October  1989,  Los  Altos  CA: 
Morgan  Kaufmann,  pp.  424-433. 

[5]  Frank  Thomeis  and  Ollie  Johnston.  Disney  Animation:  The  Illusion  of  Life.  Abbeville 
Press,  New  York,  1981. 


190 


NL  Instructions 


Natural 

Language 

Processor 

_ 

LJ 

1 

1 

Narrative  Planner 

_ \ 

&  Generator 

Plan  J 

Motion 

Generators 

I 

f  FramC'Sliced  \ 
parameter  bindings^ 

£ 


Figure  1:  Design  Framework 
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